1. Field of the Invention
The present invention relates to an image extraction apparatus and method for extracting a target subject from a background image and a subject image. More particularly, the present invention is directed to a method and apparatus for appropriately generating a mask used for extracting a target subject.
2. Related Arts
Conventionally, as general techniques for realizing image extraction, a chromakey method using a specific color background, a videomatte method for generating a key signal by performing a histogram process, difference (or differential) process, contour enhancement or contour tracking process of an image signal (The Television Society Technical Report, Vol. 12, pp. 29-34, 1988), and the like are known.
A technique for performing image extraction based on the difference from the background image is a state-of-the-art one, and for example, Japanese Patent Laid-Open No. 4-216181 discloses a technique for detecting or extracting a target object in a plurality of specific regions in an image by setting a mask image (i.e., a specific processing region) in difference data between the background image and the image to be processed.
Furthermore, Japanese Patent Publication No. 7-16250 discloses a technique for obtaining color-converted data of an original image including a background using a color model of the object to be extracted, and the existence probability distribution of the object to be extracted from brightness difference data between the background image and the original image.
In the difference method from the background image, the luminance level or color component difference between the pixels of the background image and the subject image is normally expressed by a predetermined evaluation function, and the evaluation function is subjected to a thresholding process to extract a region having a difference level equal to or higher than an initial value. As the evaluation function, the correlation between blocks having individual points as centers and a predetermined size (Rosenfeld, A. and Kak, A. C., Digital Picture Processing (2nd ed.), Academic Press, 1982), normalized principal component features (Journal of the Institute of Electronics, Information and Communication Engineers, Vol. J74-D-II, pp. 1731-1740), a weighted sum value of a standard deviation and a difference value (Journal of the Television Society, Vol. 45, pp. 1270-1276, 1991), a local histogram distance associated with hue and luminance level (Journal of the Television Society, Vol. 49, pp. 673-680, 1995), and the like are used.
Japanese Patent Laid-Open No. 4-328689 and Japanese Patent Publication No. 7-31248 disclose a method of extracting a moving object alone by extracting motion vectors or inter-frame difference data from moving images. Japanese Patent Publication Nos. 7-66446, 6-14358, and 4-48030 disclose a method of extracting a moving object based on the difference from the background image. Furthermore, a method of extracting the binocular disparity distribution (i.e., the distance distribution from image sensing means) from images from right and left different view point positions obtained using a binocular image sensing system, and segmenting an object from the background on the basis of the disparity distribution (1995 Information System Society Meeting of the Society of Electronics, Information and Communication Engineers, pp. 138), or the like is known.
However, of the above-mentioned prior arts, the chromakey method suffers from the following problems:
i: this method cannot be used outdoors due to serious background limitations, and
ii: color omission occurs.
Also, the videomatte method suffers from the following problems:
i: the contour designation must be manually and accurately performed in units of pixels, and
ii: such operation requires much labor and skill.
Furthermore, the difference method from the background image is normally difficult to realize due to the following problems:
i: the background is hard to distinguish from the subject in a partial region of the subject including a portion similar to the background,
ii: the difference method is readily influenced by variations in image sensing condition between the background image and subject image,
iii: a shadow portion formed by the subject is hard to remove, and
iv: in order to faithfully extract the boundary line between the background and subject, the background image and subject image must have considerably different image characteristics (pixel values and the like) in the vicinity of the boundary therebetween.
The technique disclosed in Japanese Patent Publication No. 7-16250 is not suitable for image extraction of an arbitrary unknown object since it requires a color model for the object to be extracted.
In either the method of extracting a moving object from moving images or the method of extracting a subject from the disparity distribution, it is generally hard to extract a subject with high precision independently of the contrast in the boundary portion between the subject and background.